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(54) DECODING METHOD AND RECORDING MEDIUM CARRYING RECORDED DECODING 
PROGRAM 



(57) When coded video and audio signals are 
decoded by a computer, the computation amount 
required per unit time is enormous. Depending on the 
performance of a CPU. therefore, it is very difficult to 
perform the decoding in real time. With respect to an 
audio, wfien all data are not output in real time, a dull 
audio output is produced or sound skipping occurs. 

In the decoding method, when a coded video or a 
coded audio is to be decoded by using a PC, a decoding 
end target time for realizing an audio real time reproduc- 
tion is compared with an actual decoding end time in 
each of decoding units. If the actual decoding end time 
is later than the decoding end target time, a video 
decoding is temporarily Interrupted, or the priority of the 
video decoding is lowered, thereby realizing an audio 
real time reproduction. 

In the decoding method, when a coded video or a 
coded audio is to be decoded by using a PC, a schedul- 
ing method is provided to more efficiently perform a 
data input, an audio decoding, and a video decoding by 
using a buffer for at least three frames. 
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Descripti h ' ' 
Technical Field 

[0001] The present Invention relates to a method of s 
decoding coded video and audio signals, and also to a 
recording medium on which a program of decoding the 
signals is recorded. 

Background Art . iq 

[0002] Rg. 1 7 is a view illustrating the configuration 
of a usual corhputer. In the usual configuration of a com- 
puter, a CPU 1701 serving as a calculating unit, a hard ' 
disk 1702 serving as a nonvolatile recording medium, 15 
and an external memory (buffer) 1703 serving as a vol- 
atile recording medium are connected to one aihother 
via a bus 1 704 

[0003] An IEEE 1394 interface which allows digital 
video and audio data to be transmitted in real time Is 20 
standardized. An IEEE 1394 terminal 1705 which is 
based on the ^ahdard will be commercially available in 
the near future. . 
[0004] Recently, as the performarice of a personal 
computer (hereinafter, referred to as PC) are Improved, 25 
video and audio processes become to be practically 
performed by software without using a dedicated appa- 
ratiis]^; ' . ' ' ' 

[0005] : Fi^. 18 is a block diagram of a method of 
decoding and reproducing a coded video and a coded 30 
audio in k PC by software: 

A cod4d audio signal which is input from an audio input- . 
ting section 1801 is input into an audio buffer 1802 
which holds a coded audio slgnal in a predetermined 
decoding unit (in this case, one frame of a video signal). 35 
The coded audio signal Which Is read out frorh the audio 
buffer 1802 is decoded by an audio decoding section 
1803, and then output from an audio outputting section 
1803. 

Similarly, a coded video signal whtbh is Input frorh a 40 - 
video inputting section 1805 is input into a video buffer 

1806 which holds a coded video sighaf in a unit of 
frame. The coded video signal which is read^ out from 
the video buffer 1806 is decoded by a video decoding 
section 1807, and then output frorh a video outputting 45 
section 1808. 

[0006] R^. 19 is a block diagram of a decoding 
method by software in the case whelrie a coded video 
and a coded audio are input as a multiplexed sighkl to a 

The multiplexed signal which Is Input' frorh a signal 
inputting section 1901 is input into a buffer 1 902 which 
holds a multiplexed signal in ia unit of franie. Ilie aiidio 
decoding section 1803 and the video defeoding section 

1 807 extract the coded audid and the coded video from ss 
the multiplexed signal held by the buffer 1902; and 
decode them, respectively. The decoded audio dhd 
video are output from an audio outputting s ctiori 1804 



and a video outputting section 1808, respectively. 
When, during a process of obtaining and decoding a 
coded signal (a coded audio signal, a coded video sig- 
nal-, or a multiplexed signal) of a. certain frame, a coded 
5 signal of the next frame is irput into tiie buffer, the 
decoding cannot be con-ectly performed. . ; . 
Therefore, a signal of the next frame cannot be input 
into the buffer unless the decoding is completed. 
Each of the operations of the audio decoding section 
7(? 1 803 and the video decoding section 1 807 is performed 
by a CPU in the PC on the basis of a programmed pro- 
cedure. The CPU performs the. audio decoding and the 
video decoding in a unit of thread serving as a basic 
process unit, and can^ perform only , one of the audio 
15 decoding and the video decoding in one thread. 

[0007] ' When video and audio are decoded In a PC 
by using the above-described conventional method, 
however, there arises the following problem. 
[0008] The number of decoding processes which 
20 are to be performed in a PC is very large. Particularly, 
processing sections in the video decoding process is 
very larger in number than those in the audio decoding 
process. Depending on the performance of a- CPU, 
thei-efore, it is impossible to decode all data of video and 
25 audio signals in real time (in the case of an NTSC sig- 
nal, t = 1001/30 [msec] or shorter). When all data of 
video and audio signals cannot be decoded in real time, 
the data are output with being appropriately decimated 
In a unit of frame in order to perform real time reproduc- 
30 tion. When an data of an audio signal are not repro- 
duced in real time, however, a dull output such as tfie 
case of a slow reproduction Is obtained, or sound skip- 
ping occurs, with tiie result that a very unpleasant 
sound is reproduced.' By contrast, witii respect to a 
35 video signal, a visually significant interference is not 
produced even when decimation is performed on about 
one frame for each two or three frames. In a conven- 
tional technique, however, scheduling is hot appropri- 
ateily performed, so that there is a possibility that frame 
40 slips occur in a larger number. When a coded signal is 
input in real time as in the case of an IEEE 1394 inter- 
face, the next frame is input before the decoding proc- 
ess of a certain frame is ended. In the conventional 
technique, therefore, the buffer cannot be updated and 
45 ^ a failure occurs. ^ - ) 
[0009] In order to prevent such inconveniences 
from occurring; a method may be employed in which the 
number of buffers is increased in a unit of frame and 
input arid output operations are performed in parallel. 
so However, this method involves a problem in that the 
increased number of buffers causes also the production 
cost of a PC performing the process to-be raised. When 
only the increase of the buffer number is conducted and 
the schedLiling of processes Is hot performed, there is a 
55 ' possibility tiiat a CPU enters an idling state. * ■ 
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Disclosure of Invention ^ y r > ; ; 

[0010]* According to the Invention, a decoding t.vj 
method and a decoding program are provided which are. . 
excellent in efficiency, which solve^ the problems of the 5 
conventional techniques, and in which the number of 
required buffers is reduced as far as possible, and r 
scheduling of the data input and decoding in a unit of 
frame is performed, whereby an audio signal is repro- 
duced in real time withotit being interrupted ard the io 
video output rate is raised: ■ • • i ' ' 

[001 1 ] The present invention of claim; 1 is a decod- 
ing method for reproducing by decoding aicoded audio ^^ 
signal every each decoding unit .which^is obtained by 
dividing said coded audio>signal in a direction o1 a time . ?5 
axis in audio:decoding, means, and by decoding a coded , 
video signal in video decoding means, wherein; ;, 

a decoding target time for each of the decoding 
units is determined, , , 20 

a reference time pf a decoding process for each of 
the decoding units is output, 
in the decoding unit, the reference time is com- 
pared with the decoding target :time; corresponding 
to the. reference time, and. if the reference time is 25 
later than the decoding target time, the process of 
decoding the coded video signal in said video 
decoding means is temporarily interrupted. 

[001 2] The present invention of claim 2 is a decod- 30 
ing method for reproducing by decoding a coded audjo 
signal every each decoding unjt which is obtained by 
dividing said coded audio signal in a direction of a time 
axis in audio decoding means, and by decoding a coded 
video signal in video decoding rneans, wherein • 35 



a decoding target time for each pf the decoding ^ 
units Is determined, 

a reference time of a decoding process for each of 
the decoding units is output, ... "^o, 

in the decoding unit, the reference time is com- 
pared with the decoding target time con-esponding 
to the reference time, and, if the reference time is , 
earlier than the decoding target time, the process of 
decoding the coded audio signal in said audio 45 
decoding means is temporarily intenrupted, and, if , 
the reference time is later than the decoding target 
time, the process of decoding the coded video sig- > 
nal .in said video decoding means is temporarily . 
.interrupted., ; ; • . 

[001 3]; The present invention of claim 3 is a decafJ- 
ing, method for reproducing by decoding a coded audiO; , 
signal every each decoding r unit which is obtained J3y. : 
dividing said coded audio signal in a direction of a time 55- 
axis in audio decoding means, and by decoding a coded 
video signal In video decoding means, wherein 



a decoding target time for each of the decoding 
units is determined, 

a reference time of a decoding process for each of 
the decoding units is output, 
in the decoding unit, the reference time is com- 
pared with the decoding target time corresponding 
to the reference time, and, If the reference tipie is 
later than the decoding target time.-priority. setting 
means sets a priority of said video decoding means 
to be lower, or a priority of said audio decoding 
means to be higher. 

[0014] : Thepresent invention of claim 4 is a decod- 
ing method for reproducing by decoding a coded audio 
signal every each decoding unit which is otstained by 
dividing said coded audio signal In a direction of a tim 
axis in audio decoding means, and by decoding a coded 
video signal in video decoding means, wherein 

a decoding target time for each of the decoding 
units is determined. ; , ; , 

a reference, time of a decoding proems for each of 
the decoding units is output, 
in the decoding unit., tfie reference tinie is cornr 
pared with the decoding target time corresponclihg 
to tiie reference timei, and. if the reference time is 
earlier than the decoding target tirrie, priority setting . 
means sets a priority of said video decoding meani^ 
to be higher; or a priority of . said audio decoding 
^ means to be lower, and, if the reference time is jater 
than tiie decoding target time, said priority setting^ 
. means sets the priority of said vW^^ 
* means to be made lower, pr tine prlprrty of sai^^^ 
audio decoding means to b^ higher. ... ^ ^ 

[0015] The present invention of clairn.,6[sic]js a 
decoding method for reproducing by decoding a coded, 
video data in a decoding unit, characterized in that 

in the case ,wl>eref : the cpded video data in ^tiie 
decoding unit is input, the coded video data is held 
in the decoding unit by an n number pf bufifers (n is 
a natural number)..,a coded video data is selected 
from said buffiBrs, and a decoding is performed by a 
video processing section, , . , 
when an input, a selection, and a decoding are per- 
^formied on the buffers while, using: an Input process 

\number indicating^^i which, one of said buffers a 
coded, yidep d^ to be Input; a video process 
number indicating from which one of said buffers a. 

i cpded.video data that is being decoded is selected; 

. and a video, prixess candidate number indicating a 

' bufifei^.whicK a coded video data that is to be 

T - selected afte)^ the video process is. ended. 

1, the.<MN^e^ in the decoding unit is input by 

Jnput^cqrit^ra^^^^^ means into a buffer corresponding 

rjothf Jrtput^R^^ 
whenihQvid^^^^ is ended, the videp process 
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candidate number at the ending timing is set to a ' 
next video process number, and a coded Video data* ' 
of a buffer corresponding to the video process can- ■ 
didate number is decoded, and ^ ^ 

when an input in the decoding unit into said buffer is s 
ended, an input process number corresponding to a" 
buffer in which the input has been just ended is set 
to the video process candidate number, and a 
number other than the video process candidate 
number to which the input process number has 10 
been just set. and other than the video process 
number at the ending timing is set as the new input 
process number. 

[0016] The present invention of claim 7[sic] is a is 
decoding rnethod for reproducing by decoding ia coded 
audio data and a coded video data in a decoding unit, 
characterized in that ' 

in the case where a multiplexed data in which the 20 
coded Video d^^^ in tiie decoding unit and the 
coded audid data in the decoding unit are multi- 
plexed is input, the multiplexed data is held in the 
decoding linit by an n nurnber of buffers (n is a nat- 
ural niimbisr). a coded video data and a coded 25 
audio data are selected from said buffers, and a 
decoding is performed by a video processing sec- 
Jiori and an au^^^ 

when an input, a selection, and a decoding are per- 
formed on the blffferswhiile using: an input process 30 
nunfTt>er indicating irrtd'whidh pne of said buffers a 
cdded video data and a coded audio data are to be 
, input; a video process riunniber indicating from 
'which one of said buffers a'coded videodata that is 
t>eing^ decoded is selected; arid a video process 3S 
candidate number indicating a buffer whjcH stores a 
coded vide9 datai that is to be selected after the 
video process is ended. 

the multiplexed data in the decoding uhit is Input by 
input cohtrolting means irito a buffer con-esponding 40 
to the input process number, 
after the input of the rnultiplexed data into said 
buffer is ended, the coded audio data is decoded by 
'[ said audio processing sectibri; . ,/ 
when a video process is ended, the video process 45 
candidate number at the ending timing is set to a 
nefxt video process number, arid a ceded video data 
of a buffer corresponding to the video process can- 
didate number is decoded', arid ; ' . ^ 
when the audio decodirig process i^ an so 

input process number corresporidihg to a bCrffer in 
which the audio decoding process has been Just 
ended is set to the Video process cahdidate ' 
number, and a number other than, the video process ^ , 
candidate number to wtii^ the inpiit pfoces^' 55 
number has been just sdt. arid other thari the yidep^ 
process number at the ending timing 
hew input procesis number. " * 



[0017] Tfte present invention of claim 8[sic] is a 
decoding method ibr reproducing by decoding a coded 
audio data and a cbded video data in a decoding unit, 
characterized in that ' ^ : 

in the case, where a multiplexed data in which the 
coded video data in the decoding unit and the 
coded audio data in the decoding unit are multi- 
plexed is input, the multiplexed data is held in the 
decoding unit by an n number of buffers (n is a nat- 
ural nuhnber), a coded video data and a coded 
audio data are selected from said'buffers, and a 
debodihg is performed by a video processing sec- 
tion and an audio processing section. - - - ^ ' 
when ian inpiit, a'selectibn, and a decoding are per- 
forrned on the bufters while using: ah input process 
number indicating into which one of said buffers a 
coded video data and a coded audio data are to be 
input; a video process number indicating from 
which one of said buffers a coded video data thait is 

' being decoded \k selected; an audio process 
nurnber indicating from which one of siaid buffers a 
coded audio data that is being decoded is selected; 
and a video process candidate number indicating a 
tHjffer which stores a coded video data that is to be 
selected after the video process is ended, 
the multiplexed data in the decoding unit is input by 
input controlling means into a buffer corresponding 
to the input process number, 
after the input of the multiplexed data into said 
buffer is ended, the coded audio data is decoded by 
said audio processing section, 
when a video process is ended, the video process 
candidate number at the ending timing is set to a 
next video process number, and ia coded video data 
of a buffer corresponding to the video process can- 
didate number is decoded, and 
when an input in the decoding unit to said buffer is 
ended, an input process number corresponding to a 
buffer in which the input has been just ended is set 
to the audio process number and the video process 
candidate number, and (1) a number other than the 
audio process number to which the input process 
number has been just set, and other than the video 
process nijrriber at the input ending timing, or (2) a 
number other than the video process candidate 
number to which the input process number has 
been just set and other than the video jDrocess 

' number at the input process tirriihg is set as the hew 
input process number. 

[0018] ■ The present invention of claim loisic] is a 
decoding method for reproducing by decoding a coded 
audio data in an audio decoding unit, wherein 

in tiie case where the cod^ audio data in the ^dio 
^ decoding unit is held by an m number of buffers (m 
is a natural number}! the coded audio data is' 
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selected from said buffers., §nd,a,decocling is per- 
formed by an audio processing section,, ^ . s ^ 
when an input, a selectiort, and a decoding are per- 1 ,., 
formed on ttie buffers while using: an input process = ^ 
number indicating into which one of said buffers a s 
r coded audio data is to be input; and an audio proc- 
. ess number indicating from which one of said buff- 
^ , ers a coded audio data that is being decoded is 
selected, ■ , , ^ 
when an audio reference signal which is produced io 
at time intervals corresponding to frie audio decod- 
ing unit is produced. j.ari aidio buffer corresponding 
to ttie audio, inpgt nuniber ispausedito start.an input 
of the coded audiojdata, , . ^ , , . \ , .. ^ ; 
. the coded audio data is sejeded from, an^^ is 
, buffer of the^ audio , process number, the coded 
, audio data is decoded by said audio . processing 
section, : . . 

when a coded audio input in the decoding unit is 
ended, ian audio input number corresponding to a 20 
buffer in which the input has been just ended is set 
to the audio process number by process number 
updating means, and in place a number other than 
the audio process number to which the audio input 
. number has been just set is set as the new audio 25 
input number. 

[0019] The present inverttion of claim 11 [sic] is a 
decoding method for input and reproducing by decoding 
a coded data in a decoding unit, wherein 30 

in the case where the coded data in the decoding 
unit is held by an m number of buffers (m is a natu- 
ral number), the coded data is selected from said 
buffers, and a decoding is performed by a process- 35 
ing section, 

when an input of a coded data into said buffers is 
ended, numbers of kxiffers in which the input is 
ended are accumulated into a process carxjidate 
list, when a coded data is selected for decoding 40 
from a predetermined buffer, a number of the 
selected buffer is deleted from the process candi- 
date list. \ . . , 
when a process of a coded data is ended, among 
buffer numbers accumulated in the process candi- 45 
date list, a buffer number an accumulation time of 

. which is earliest is selected, and a decoding is per- 
formed by a processing section, and 

. when the coded data is to be input, a number of 
buffer numbers which are accumulated in the proc- so , 
ess candidate list is checked, and, if the number 
reaches a predetermined number a which is not., 
larger than, the number in of said buffers, the coded' , . 
data is not input, and, if only buffer numbers^ a 
number of which is smaller than a are aiccurnulated; ss 
the coded data is input. . , 

[0020] The, present invention of claim 17[sic] is a 



program , recording meciium characterized in that a pro- 
gram Jfor realizing all or a part of steps 'the decoding 
method according to any one of claims 1 to .17 by a 
computer is stored in said medium. 

Brief Description of Drawings 

[0021] ; , 

Fig., 1 is a block diagram illustrating a first embodi- 
ment of the invention, r 
Fig. 2 is a block diagram^ illustrating a second 
errtoodiment of the inverttion. - 
Rg. 3 is a block diagram illustrating a third embodi- 
merit of the invention. , . . 
Rg. 4 is a block diagram itlustratihg a tourth embdd- 
irnent of the invention. 

pig. 5 is a block diagram illustrating a fifth ernbodi- 
ment of the invention. 

Rg. 6 is a timing chart illustrating a , decoding 
method in the fifth embodiment of the invention. 
Rg. 7 is a block diagrarfi illustrating a sixth. embodi- 
ment of the invention. . , . - . 
Rg., 8 is a timing chart illustrattrig a deccxjing 
method in the sixth embodiment of the invention. 
Rg. 9 is a block diagranh illustrating a seventh 
embodiment of the invention. . / 
Rg. io is a timing c^art illustrating a decoding 
.method in the seventii enribodirrient of the Irw^ntion. 
Rg. 11 Js a block diagram illli^tratihg an. eighth 
embodiment of frie inverttion, 
Rg. 12 is a timing, chart illustrating a decoding 
method in the eighrth enTbbdimerit of the invention. 
Rg. 13 is a blod^, diagram illListrating a ninth 
embodiment of the irivention. l 
Rg. ,14 is a timing chart illustrating a decoding 
method ih the njnth embodiment of the invention. 
Fig. 15 is a block diagram illustrating a tenth 
ernbpdirrjentpf theirrventibh.,, . 
Rg, 16 is a; timing chart illustrating a decoding 
method in the tenth embodiirient of the invention. 
Rg. 17 is a^schematic view illustrating the configu- 
ration of. a lisuaj PC. 

Fig is is a block diagram illustrating a conventional 

method of decoding a coded , video and a coded 

audio, ,^ ' 

Rg. 19 is'a block diagram illustrating a conventional 

niethod of decoding a coded video and a coded 

audio which are rnultiplexed. 

Fig. 20 is a tirning chart showing the operation of 

the. embodiment o^ 

Best jMpde for Carrying Out the Invention 

. [OO221I Z hl^reihafter.' embodiments of the invention 
wiir be 'y^d^ri^ yi/'tth reference to the drawinigs. In the 
embckjinnen^ as th t^asic time unit for 

video arid audio decoding processes, one frahrie period 
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of an NTSC signal of t = ioo 1/30 [msec!] is used; It is 
assumed that all the procedures of prdcesses wNch will • 
be described in the embodiments are described as pro- - ^ 
grams for a PC, calculation is performed by a CPU, and 
video and audio decodings are performed in different 5 
threads. 

[0023] Fig. i is a block diagram Illustrating a first 
embodiment of the invention. In Fig: 1. 101 denotes an 
audio inputting section, 102 denotes a video inputting 
section. 103 denotes an audio decoding section, 104 10 
denotes a video decoding section. 105 denotes an 
audio outputting section. 106 denotes an audio decod- 
ing completion time, 107 denotes a process interrupt 
request signal producing section. 108 denotes a target 
time producing section, i 09 denotes a target time. 1 10' is 
denotes a video decoding interrupt request signal, and 
111 denotes a video outputting section. 
[0024]' ' Hereinafter, the operation of the first emlxxJ- 
iment wjll be described. 

[0025] A coded audio signal which is input from the 20 
audio inputting kection 101. and a coded video signal 
which is iriput from the video inputting section 102 are 
decoded by the audio decoding section 103 and the 
video decoding section i 04. respectively. 
[0026] ' The audio" decoding section 103 transmits 25 
the decoded audid to the audio outputting section 1 05 to 
reproduce the audioi and notifies the process interrupt 
request signal producing section 107 of the audio 
decodirig cbmpietion tirrie 106. 

[0027] On the dthe^r hand/since the audio reproduc- 30 
tion is performed in real time, the target time producing 
section 108 produces tiie target time 109 for the audio 
decoding completion fbr each frame, and notifies the 
process interrupt request signal producing section 107 
of the produced target time. When the time of ^rting 35 
the process is indicated by s; the tkrget time when the 
audio decoding process for an n-th frame is complete^ 
iss + tn. ■ ' ' ■ _ y ' ' ' ' ' 
[0028] The process inWript request Signal produc- 
ing section 1 07 compares the targiet time 109 produced 40 
by the target time prddudng section 108 with the audio 
decoding completion tirne 106, and detects for each ■ 
frame Whether the decoding process is delayed of not! 
[0029] If the audio decoding completion tirne i 06 is' 
later than the target ti me 1 09 (if the conpletiori report is " 45 
not received before the target timiB, it is determined that 
the audio decoding completion time lags), the process 
interrupt request signal producing section 107 transmits 
the video decoding interrupt request signal iio to the 
video decoding section 104. When the vicleo decoding so 
section 1 04 receives the video decoding interrupt 
request signal 110. the video decoding section 104 ' 
interrupts the video decoding process. At this time,' the " 
CPU can allocate more resources Xo the^^udio diecoding ^ 
process. As a result; completion of the subsequent' 55 
audio decoding process can be rrtade eariier than the *; 
target'time. ^ ' "' ■''■/-■ -' "''y' 

[0030] If the process interrupt request signal pro^^ " 



ductng section 107 detects that the audio decoding 
completion time 106 is earlier than the target time 109, 
the process interrupt request signal producing section 
107 stops the' transmission of the video decoding interT 
rupt request signal 1 10. At this time; the video decoding 
section 104 starts decoding of a video signal of the 
frame in which the audio decoding has been just com- 
pleted, and the decoded video is output from the video 
outputting section 111 . , 
[0031] As described above, in the first embodiment 
of the invention, when completion of the audio decoding 
lags behind the target time, the video process is inters 
rupted and the audio decoding process is preferentially 
performed, whereby the audio signar is enabled to be 
reproduced in real time. 'V , / v: 
[0032] Depending on the polormance of the GPU, 
when the transmission of the video decoding interrupt 
request signal 1 10 is stopped, the video decoding sec- 
tion 104 may start decoding of a video signal of a frame 
succeeding that in which the audio decoding has been 
just corhpleted, whereby the video and- audio signals 
can be synchronized with each other ^ ^ 
[0033] In order to illustrate the flow of the above- 
mentioned operation. Fig. 20 shows a timing chart of 
the operation. 

[0034] Fig. 2 is a block diagram illustrating a second 
emt)odiment of the invention. In Fig. 2. 201 denotes an 
audio decoding interrupt request signal. 
[0035] Hereinafter, the operation of the second 
embodiment will be described with paying stress on por- 
tions which are different from the first embodiment. 
[0036] In the same manner as the first embodiment, 
the audio decoding section 103 produces the audio 
decoding completion time 106. and the target time pro- 
ducing section 108 produces the target time 109. 
[0037] If it is detected that the audio decoding com- 
pletion time 106 is earlier than the target time 109. the 
process interrupt request signal producing section 107 
transmits the audio decoding Jnten-upt request signal 
201 to the audio decoding section 103 till the time 
reaches the target time 109. During a period when the 
audio decoding interrupt request signal 201 is received, 
the audio decoding section 103 interrupts the audio 
decoding process. At this time, the GPU can allocate 
more resources to the video decoding process. As a : 
result, the video reproduction rate can be improved. 
[0038] If the process interrupt request signal pro- 
ducing section 107 detects that the audio decoding 
completion time 1 06 is later than the target time 1 09; in 
the same manner as the first embodiment, the process 
interrupt request siignal producing isection 1 07 transmits 
the video decoding interrupt request signal 110 to the 
video decoding 'section 104. to iritierrupt the video 
decoding process, so that the audio output is performed 
in real time: 

[0039]' As described above, in the embodiment of 
the invention, when the decoding of an audio signal is 
completed earlier than the target time, the audio decbd^ 
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ing process is interrupted till the time,reaches1tie target 
time. and. during this period, the CPU resource is alio- . 
cated to the video decoding process, whereby the audio, - 
signal is enabled to be reproduced in real time and the 
video reproduction rate can be improved.. , . 5 

[0040] In the embodiment, the transmission period ■ 
of the audio decoding interrupt request signal 201 is set 
to extend from the audio decoding completion time to 
the decoding completion target time. The transrnission 
period of the audio decoding interrupt request signal , 70 
201 can be freely set in a range, where the real time 
property of the audio output is not impaired. .In other ; 
words, the interruption period may be shortened , 
interruption^may be; rhacjejonger than theitarget;;;^ - k, , ' 
[0041] Fig. 3 is a block diagram illustrating a; third is 
embodiment of the invention;, In Rig. 3, 301 denotes, a , 
priority setting section,: and 302 denotes a iVideo decod- 
ing priority. . ^ . . ^ 
[0042] Hereinafter, the , operation : pf the third 
embodiment will be described with paying stress on por- 20 
tions-which are different from the first embodiment. 
[0043] In the embodiment for the sake of simplicity, 
it is assumed that the priority has either of three steps of 
0, 1. and 2. and, as the numeral is larger, the process 
priority in the CPU is higher 25 
[0044] Prior to start of the decoding process; the 
process priority in the CPU is set for each of the audio 
and video decodings. For example, as the initial set 
value of the priority, the audio decoding is provided with 
2. and the video decoding with 1 . , 30 
[0045] In the same manner as the first embodinient. 
the audio decoding section 103 produces the audio 
decoding completion time 106, and the target time pro- 
ducing section 108 produces the target time 109. 
[0046] If the priority setting section 301 detects that 35 
the audio decoding completion time 1 06 is later than the ^ 
target time 1 09, the priority setting section 301 reduces 
the video decoding priority 302 to 0, thereby making the, 
priority of the audio decoding relatively higher. The CPU 
performs scheduling of processes in accordance with 40. 
their priorities. In this case, the audio decoding process 
is preferentially performed, and hence the real time 
property of the audio decoding can be ensured. , 
[0047] If it is detected thai the audio decoding com- 
pletion time 106 is earlier than, the target timer 109. the 45 
video decoding priority is returned from 0 to 1 by the pri- 
ority setting section 301 .. 

[0048] V • . As described above, in the third embodiment ^ 
of the invention, when completion of the audio decoding 
lags behind the target time, the priority of the video sig- so 
nal is lowered so as to relatively raise the priority of the 
audio decoding, so that the audio decoding i^ preferen- j, 
tially performed in the CPU thereby enabling the audio 
to be reproduced in real time. , . - . ^ ^ ^ 

[0049] Fig. 4 is a block diagram illustrating a fourth 55. 
embodiment .of the invention: In Fig. 4, 401 denotes an v 
audio decoding pri rity. : ■ . = - , , 

[0059] Hereiriafter, the operatipn of the ifpurtK- 



enriljpdimerrt will be described with paying stress on por- 
tions which are different from the third embodiment. 
[0051] , In the embodiment also, it is assumed that 
the priority has either of three steps of 0, 1 , and 2, and, 
vas the numeral is larger, the process priority in the CPU 
is higher. Prior to start of the decoding process, the ini- 
tial set values of the priorities of the audio and video . 
decodings are $et to 2 and 1 , respectively 
[0052] In the same manner as the third embodt: 
merit, the audio decoding section 103 produces the 
audio decoding completion time 106. and. the target 
time producing ^section 108 produces the . target, time 

[0053] i If the priority setting section 301 detects that 
the audio decoding completion time 106 is. earlier than 
the target time 109. the priority setting section 301 sets 
the audio decoding, priority 401 to 0. At this time, the 
CPU preferentially performs the video. decoding proc- 
ess, with the result that the video reproduction rate can 
be improved. When the time reaches the target time 
109, the audio decoding priority 401 is returned to the 
initial set value or 2. , * , 

[0054] If it is detected that the audio decoding com- 
pletion time 106 is later ,than the target time 109. iri the 
same manner as the third embodiment, the priority set- 
ting section 301 reduces the video decoding, priority 
302. thereby enabling the real time property of the audio 
decoding to be ensured. . _ « ; 

[0055] As described al>ove, in the emtji^JiiTient^ o 
the invention, when the decoding of an audio, signal is. 
completed earlier than the target time, the audio decod-. 
ing priority is lowered arid the video decoding priority Js 
raised till the time reaches the target tiirie, thereby caus- ; 
ing the CPU to preferentially, perfprm^ti^^^ 
ing process. As a result, ttie video reproduction, rate can 
be jftprpved while paintairiing the real time output of 
the audio. ^ L - r,' -.^ ^ 

[0056] In the embodiment, when the audio decod- 
ing is completed eariier .than the target time, the audio 
decoding priority is lowered. As far as the audio decod-; 
ing priority is relatively lower and the video decoding pri- 
ority is relatively .higher, alternatively t^ niay 
be selected jn any manner The period ^when the audiq. 
decoding priority, is made relatively lowered extends to 
the target time. This period' may be freely set in a range 
where the real time property of the audio ou^ut is not 
impaired.. , 

[0057] . Jn the third and fourth embodiments, the pri- 
ority has three Sfteps- Aiternatiyely the priority rnay hav 
. any nun^er of steps,^^ and may/have coritinuous values 
in piace/ofJhe discrete values as thc^^^ of the embddir 
meijts. ,^ V IS f ■> . - ■ • ■ 

[OOSS] Fig. 5 Js a block diagram . illustra^ 
ernbodirnent of the invention. Jri Fig.,5. 501 denotes a 
data inputting se^^ an input controlling 

sectiprj..,5^^^ setting section, 

504 denotes a buffer,' 505 denotes a video buffer select- 
ing sectipri, 551,, denotes an ^ input process number. 552 
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denotes a video process number, and 553 denotes a 
video decoding end signal. ' : Ni'w . rr = 

[0059] in the subsequent embodiments/ h "^ is' ' 
assumed tliat tlie buffer 504 lias three sub-buffers of ' 
buffer numbers 0, 1, and 2. Each of the sub-buffer=s 5 
stores a data of one frame, tt is assumed that the data;^ 
Input and the video decoding operate in different 
threads. 

[0060] Next, the operation of the embodiment will 
be described. io 
[0061] For a coded video data which is input from 
the data inputting section 501 . the input controlling sec- 
tion 502 designates the sub-buffer corresponding to the 
input process number 551 which is supplied from the 
buffer number setting section 503. The coded video is 
data is held by the buffer 504. 

[0062] After a data of one frame are inpiit, the video 
buff r selecting section 505 extracts a coded video data 
from the sub-buffer corresponding to the video process 
number 552 supplied irbm the buffer number setting 20 
s ctibn 503. and supplies the extracted data to the 
vid b decoding section iOi4. 

[0063] In the video decoding section 1 04, the coded 
video is decoded ^ and then output from the video out- 
putting section 111. When the video decoding for one 25 
frame is ended, the video decoding end signal 553 is 
supplied to the buffer nurhber setting section 503. 
[0064] Rg. 6 is a tirhe chart illustrating selections of 
the input process number and the video process 
number in the buffer number setting section 503 of the 30 
embodiment, in Rg. 6, a video process candidate 
number is the hUnidjier of the sub-buffer in which, when 
the current video decoding is completed, the frame to 
be next decoded ii'held, and the video process number 
is the number of the sub-buffer which is currently sub- 35 
jected to the video process. 

[0065] For the^input controlling J section 502 of Fig. 
5» the input pirocess number is the niinriber of the sub- 
buffer which is td%e cunrently subjected to a writirig 
process. .^^ 
[0066] In the buffer niJhit>er setting section, the 
video process candidate nunriber is set in addition to the 
input process number and the video process nuhrtber 
corresponding to the sub-buffer 0, 1, or 2. All the initial 
values of these numbers are set to "0.'' ' 45 
[0067] AttimeTO, the input of a data of a 6-th" frame • 
is started, and the data is input into sub:buffer 0 corre- 
sponding to the Input process number 6. When the input 
of the 0-th frame is completed at time T1, the video 
process candidate number is updated to "0" that is the so 
buff r number in which th!e input* has been just com- ' 
pleted, and a new input process number is set to "sub- 
buffer 1 (either of "V) and.*'2'' is availabfe. aihd^^ 
case, *'1'' is provisionally is set for the sake of cphven- ^ - 
ience in description) that is a nunrbei- other than' the '5^ 
video process candidate number (i:e:. oth6r th^n' "^ ' 
and other than the video process riurnber (i.W!? qther^ ^ 
than "0"). • ' ' 



[0068] ' At '-the same tiine; "0" which is the video 
process candidate number is set as the video process 
number In the video buffer selecting section; the coded 
video is obtained from sub-buffer 0 corresponding to the 
video process number 0, and the Video decoding proc- 
ess is started. 

[0069] At time T2, a coded video of a third frame is 
input into sub-buffer 0 corresponding to the input proc- 
ess number 0' ' ■ 

[0070] " When the input of the coded video of the 
third frame is ended at time T3, "0" which is the input 
process number is set as the video process number, 
and 1 that is a number^ other than the video process 
number at time T3 (i.e., other than **2") and diher than 
the updated video process candidate nunriber (i;e.. other 
than "0") is set as a hew input process number: 
[0071] Whentheir^utof the coded video of a fourth 
frame into buffer 1 is completed at time T4, "1" which is 
the input process number is set as the video process 
candidate number, and "0" that is a number other than 
video process candidate number "1 " and the video proc- 
ess number 2 at time T4 + AT (where AT is a minute 
time period) is ^ designated as a new iriput process 
number ' i . 

[0072] When the video decoding of the second 
frame which has been stored in sut>buffer 2 is ended at 
time T5. "1" which is the video process candidate 
number at time T5 is set as the video process number, 
and the decoding of the fourth frame which is held by 
sub-buffer 1 at this timing is startied. 
[0073] In the case of Fig. 6, the thihd frame is not 
decoded. In the video decoding, even when about one 
frame slip occurs in every several frames, a visually 
serious problem is not produced. 
[0074] As described above, according to the 
embodiment, when a fcxjffer for at least three frames is 
prepared, a buffer which is being subjected to the video 
decoding process is prevented froni being ovenwritten 
with a new coded data. Therefore, the video reproduc- 
tion can be performed at a rate as high as possible in 
accordance with the ability of the CPU. 
[0075] In the embodiment described above, sub- 
buffers for three frames are prepared. Also when sub- 
buffers for four or more frames are prepared, the same 
effects as those of the case where sut>-buffers for three 
frames are prepared are attained. - ' 

[0076] Fig. 7 is a block diagram illustrating a sixth 
embodiment of the invention. In Fig. 7, 703 denotes a 
buffer number setting section, 705 denotes an audio 
buffer selecting section, and 751 denotes an audio 
decoding end signal. 

[0077] In the ernbodiment also, in the same rhanher 
as the first embodiment, it is assurhed that the buffer 
504 has three sub-buffers of buffer numbers 0, 1 , and 2'. 
' In the embodirhent. it is assumed that the inputting proc- 
ess and the audio process are performed in the same 
thread and th^ audio process is started immediately 
after the input of a data in which s^deo and audio data of 
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one frame, are multiplexed, .irito the sub-buffers . is , ,^ 
ended. Therefore, a sut>bLjrffer .of .the. same buff^ -..^ 
number contains video and , audio, data of the^same . ; 
frame, f^urthermore, it is assurned that the video proc-» 
ess is performed in a thread which is different from the. . 5 
thread for the inputting process and the audio process. . 
[0078] Next, the operation of the ernbodiment will ^ 
be described. . ^ > : - , ^ . ■ ^ r 
[0079] In the input controlling section 502. a data , 
which is input fram the data inputting section 501 and in = 10 
which a coded video and a coded audio are multiplexed 
is held by the sub-buffer cpn-esppnding to the input 
process njumber 551 which is supplied from; t^ ^ 
number setting section j7p3. v - ' -r r. ,rj. ; 
[0030] . After a dataqf pnelrame is ir^^ the^audio . is 
buffer selecting section 705 extractsa coded audio data 
from the sub-buffer corresponding to the input process 
number 551 supplied from the buffer number setting 
section 7303,. i.e., the sub-buffer in which the data input 
has been just completed, and supplies the extracted 20 
data to the audio decoding section 1 03. 
[0081] In the audio decoding section 103, the coded 
audio is decoded, and tiien output from the audio out- 
putting section 105. When the audio decoding is com- 
pleted, the audio decoding end signal 751 is supplied to 25 
the buffer number setting section 703. 
[0082] The video buffer selecting section 505 
extracts a coded video data from the sub-buffer in the 
buffer 504 and corresponding to the video process 
nunnber 552 supplied from the buffer number setting so 
section 703, and supplies the extracted data to the 
video decoding section 104. In the video decoding sec- 
tion .104, the coded video is decoded, and then output 
from the video outputting section 111. When the video 
decoding for one frame is ended, the video decoding 35 
end signal 553 is supplied to the buffer number setting 
section 703. 

[0083] Fig. 8 is a time chart illustrating the setting of 
the input process number and the video process 
number in the buffer number setting section 703 of the. 40 
embodiment. In Fig. 8. a video process candidate 
number is the number of tiie sub-buffer in which, when 
the current video decoding is completed, the frame to 
be next decoded is held, and the video process number, 
is the number of the sub-buffer which is currently subr 45 
jected to the video process. - 
[0084] , For the input controlling section 502 of F^ig. 
7. the input process number is the number of the sub- 
buffer into which a data of a frjame is to be written. For 
the .audio buffer .selecting section, the input process , so 
number is the number of the sub-buffer from which, ,3 
multiplexed audio data in a data of a frame is read. ^ 
[0085] ^ In. the buffer number setting, section, the 
video process candidate number is set in ajddition tojhe . 
input process number and the video process number. . 55 
All the initial values of these numbers are set to "0." 
[0086] As described in the Back Ground, section, 
required calculatipn in an audio decoding process is, . 



usually^smaller than that in a video decoding. prpcess, 
and, with respect to a decoding process time period for 
one. frame,, an audio is shorter than a video. In order to 
prevent dull audio reproduction or sound skipping from ^ 
occuWing, the priority of the audio decoding process 
threadjs sometimes set to be higher than that of the 
video decoding process thread. 
[0087] At time TO , a data of a 0-th frame is input into 
sub-buffer 0 corresponding to tiie input process number 
0. When the input is completed, a coded audio is imme- 
diately extracljed from sub-buffer O^in which the input 
has been just completed, and the audio decoding proc- 
ess Is started. 

[0088] In response to the receptipn of the audio 
decoding end siignal from the audio decoding section, 
the buffer numt)er setting section ' 703 , imnrtediately 
inputs the input process number into the video process 
candidate number, and, as a new input process nunilaer, 
sets a nuntber other than tiie video process number and 
the updated video process candidate number.. 
[0089] In this case, as the ^ new input process 
number, a number other than the .video process number 
is selected in order to prevent overwriting from occurring 
in a sub-buffer which is being subjected to the video 
process at that timing. As the new input process 
nunrtjer. a number other than the updated, video proc- 
ess candidate nunf*>er is selected in order to prevent. 
ovenA^riting from occurririg in a sub-buffer storing a data 
which is scheduled to be next subjected to ^he video ' 

process. - ^ 

[0090] When the video decoding end signal is 
received from the video decodirig section, the;video 
process candidate number which .is tiie t>uffer numtper, 
where a data that is scheduled tp be^ next subjected .to 
the video process is set. as a new ^ video process 
number. 

[0091] , When, the audio deccding of the first frame 
extracted from sub-buffer 1 |s ended at time T1 of Fig. 8, 
"1" which is the immediately, ^ previous jnput process 
V number is set as tiie video process candidate nunnber. , 
[0092] At T1,+ .AT (where AT is a minute tirne 
period), the input process nuniber is "1," and the video 
process^ nurriber is."6," Therefore, "2" is set as the new 
input process number.. 

[0093] Then. attimeT2, the end of the video decod- 
ing of the 0-th frarrte extracted from buffer 0 is notified. 
Therefore,^ 'Vl" which is. the video process candidate 
nurnber at time T2,is set as the , new video process 
nurriber. . 
[0094] At time T3. similarly,, the end of the audio 
decoding of , the third frarhe^^^ from sub-buffer 0 
is notified* and "9" Whjch is the immediately previous 
Inpirt process number is_set as the video process candi- 
date nurriber. At;time f3 + AT (where ATJs a minute, 
time period). , the input process number is "0." and the 
video process numt>er is "2." Therefore, the new input 
process riumber js,"1" which Is a value other than "0" 
and'"2"; ' ' ' ' , , 
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[0095] In tiie embodiment when the audio decod- '■ ' 
ing end signal is transmitted plural times (times T3 4ritf- 
T4) during a certain video decoding period as iniKe" 
case of the second frame of^Rg. 6, frame skippiiig 
occurs in the reproduced video. ' " ' s 

[0096] With respect to a video signal, however, a 
visual problem is not produced even when afcx)ut one 
frame skipping occurs in every several frames. 
[0097] As described above, according to the 
embodiment, a buffer for at least three frames is pre- 'io 
pared, and the buffer management described in the 
embodimerrt Is conducted, whereby the audio decoding ' 
can be input without lacking any frame, and a buffer 
which is being subjected to the videb decoding process 
is prevented frorh being overwritten with a new multi- is 
plexed data. Therefore, sound skipping does not occur 
during the audio reproduction, and the video reproduc- 
tion can be performed at a rate as high as possible in 
accordahcie W'rth ttie ability of the CPU. 
[0098] If a buffer for three frames is simply held and 20 
the buffer operatibh of the embodiment is not per- 
formed,' there arises a possibility that overwriting Is per- 
formed on a buffei-^oWrng a data which should be 
originally next reproduced, and hot only reduction of the 
image reproduction rate but also missing of the audio ss 
reproductlori occurs. - - 

[0099] The decoded audio signal of one frame is 
asynchronous. Therefore, when' the audio butputting 
sectibri has a buffer and an audio output reference sig- 
nal, and frames are reproduced one by one in accord- so 
ance with the audio output i^efererjce signal, the audio is 
correctly reproduced withbut being dulled. When the 
audio output reference signal Is supplied also to the 
video outputting section, the video and the audio can be 
output in synchrbnizatibh with each cither. 35 
[0100] In the embodiment; wherl the inputting proc^ 
ess is ended, the audio decbdinicj' process is immedi- 
ately started. The invention is not restricted to this as far ' 
as t>oth the inputting process and the audio decoding 
process can be end^ within one frame period. '40 
[0101] in the embodiment described above, sufc)- 
buffers for three frames aire prepared. Also when sub- 
buffers for four or more frames are priepared, the s^me ' 
effects as those of the base where dub-buffers for three 
frarhes are prepared are attaiined: ^' " 45' 
[0102] Fig. 9 is a viw illustrating^W seventh embod- 
iment of the invention. In Fijg. 9, 903 denotes a buffer 
number setting section. 95t denbtfes an input end sig- ' 
nal, and 952 denotes an audio buffer number. - 
[0103] In the embodiment aliso. in thfe same mahnbr ^ so 
as the fifth embodiment, it i^ assurhed that the buffer 
504 has three sub-buffers of buffer numt>ers O. i, and 2: ' ' ' 
In the embodiment, it is assumed that the inpiJttingiJroc- 
ess, the audio process, and the video process arie 
respectively performed in different threads. ' ' ' 'ss' 
[0104] N©ct, the operation of the'' embodiment will " 
bedescribed: • ^ ■ 'y^--'^'-- v--^^.-', . 

[OiOS] In the Input controli ing sebtibn 502, a datia' 



which is input from the data inputting section 501 and in 
which a coded video and a coded audio are multiplexed 
is held by the sub4>uffer corresponding to the input 
process number 551. i, ^ - . 

[0106] When the input of a data of one franie is' 
conipleted. the butter 504 transmits the input end signal 
95t to the buffer number setting section 903. 
[0107] In the buffer number setting section 903, 
when the input ehd signal 951 is received, the input 
process number is set to ttie audio process number 952 
and the video process candidate number, and the audio 
process number 952 is immediately transmitted to the 
audio buffer selecting section 705. ' * ' - ' 
[0108] ' A huiTTber-^other 'tr'an^'t process 
number arid the audio process numbe is set as 'a new 
inpiit process number. TTie audio buffer selecting sec- 
tion 705 extracts a coded audio data from the sub^buffer 
corresponding to the audio process signal 952, and 
supplies the extracted data to the audio decoding sec- 
tion 103. - 

[0109] The video decoding operates in the same 
manner as the sixth embodiment. 
[01 10] Fig. 10 is a time chart illustrating selections 
of the input process numbier, the audio process number, 
and the video process number in the buffer number set- 
ting section 903 of the embodiment 
[01 1 1 ] At time TO; the input of a multiplexed data of 
a 0-th frame into sub-buffer 0 corresponding to the input 
process number 0 is started. When the Input is com- 
pleted at time T1 . the input ehd signar951 is transmitted 
from the buffer to the buffer number setting section. "0" 
is set to the audio process number and the video proc- 
ess candidate number, and 

a number (in this case, "1 ") other than the video process 
number "0" and the audio process number "0" Is set as 
a new input process number. At the same time, a coded 
audio is extracted from sub-buffer 0 and the audio 
decoding process is started. 

[0112] Furthermore, the video process candidate 
number 0 is immediately set to the video process 
number, the coded video is extracted from sub-buffer O; 
arxl the video decoding process is started. 
[0113]' At tinrie T2, the input of a multiplexed data of 
a second frarhe into sub-buffer 2 is started. 
[01 1 4] Wheri the input of the multiplexed data of the 
second frame into sub-buffer 2 is completed at time T3, 
"2" which is the buffer number where the input is com- - 
pleted is set to the audio process number and this video 
process candidate number, and the audio decoding of 
the second franrie' is immediately started. Furthermore'. 
"0" 'other than the video process humber "r and the 
audio prbcess number "2*' at time T3 is set as a hew 
input process number. ^ 
[0115]; By contrast, at tinie T4, 'tfie video decoding 
of a first frame iextracted from sub-buffer i is ended. At 
this time, the video process candidate number ^"0" at 
time t4 is set as the video process niimber, and, in^the 
next video decoding process, the decoding of a third^ 
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frame held by sub-buffer 0 is started. , - ^ ; v • , 
[01 1 6] ' As desaibed above, according' , to the v 
embodiment, a buffer for at least three frames is pre- .H 
pared, and the buffer management described in the 
embodiment is conducted, whereby the audio decoding ■ s 
can be performed without jacking any frame, reproduc- 
tion is possible without any sound skipping, and a buffer ^ 
which is being subjected to the video decoding process - 
is prevented from being overwritten with a new coded , 
data. Therefore, the video reproduction also cari be per- io 
formed at a rate as high as possible in accordance with, 
the ability of the CPU. ; - . . . ; - , 

[0117] If a buffer for three frames is simply held and . 
the buffer oporatipni of .th? ernbodirnent is not per- 
formed, there arises a possibility' that ovenwriting^is perr . ys 
formed on a . buffer; holding, a data ^ which should be . 
originally next reproduced, and not only reduction of tfie 
image reproduction rate but also missing of the audio 
reproduction occurs. 

[0118] The seventh embodiment has a configura- 
tion in which the audio process signal 952 is transmitted 
from the buffer number setting section 903.^ Alternar, 
tively. a configuration in which the signal is directly ^ 
transmitted from the buffer 504 may be employed. . : 
[0119] ln,ftfie emtjodiment, the audio decoding is . 25 
started immediately after, the input of one frame is 
ended. The invention is not restricted to this as far as 
the audio decoding process can maintain the real time 
property. , . , 

[0120] In. the embodiment, sub-buffers for three 30 
frames are prepared. Also when sub-buffers for four or 
more frames are prepared, the same effects as those of 
the case where sub-buffers for three frames are pre- 
pared are attained. 

[0121] Fig. 11 is a view illusfrating an eighth 35 
embodiment of the invention. In Fig. 11,1101 denotes a 
reference signal producing section, and 1 151 denotes a 
reference signal. 

[0122] Next, the operation of the embodiment will . 
be described with paying stress on portions which are 40 
different from the seventh embodiment. 
[0123] In the embodiment, in the same manner as 
the seventh embodiment, it is assumed that the input- 
ting process, the audio process, and the video process 
are respectively performed in different threads. , 45 . 

[0124] - The reference signal producing section 1101 
supplies the reference signal 1 151 to the input control- 
ling section 502, once in one frame period (in the case 
of an NTSC signal, about 1/30 sec, or rnore correctly 
1001/30 msec). \A/hen the reference signal 1151, is . so 
input, the input controlling section 502 starts to obtain a 
multiplexed data from the data inputting section 501, , 
and supplies the multiplexed data to the sub-buffer cor- ; 
responding to the input process number 551 which is 
supplied from the buffer number setting section 503., ' ss 
When the input of a data of one frame is completed, th^ 
buffer; 504 fransmits the input end signal 951 to the ^ 
buffer number setting section 903. . ; . 



[0125] t :^ In the buffer nun^er setting section 903, 
when, j^e; input end signal 951 Is received,, the input 
process nurnber is set to the audio process number 952 
and, the video process candidate number, and the audio 
process number 952 is immediately transmitted to the 
audio buffer selecting section 305. A number other than 
the video process number and the audio process 
number is set as a new input process number. The 
audio buffer selecting section 705 extracts a coded 
audio data from the sub-buffer corresponding to tfie 
audio process signal 952. and supplies the'exfracted 
data to the audio decoding section 103. ^ 
[0126] j The video decoding operates in the same 
manner as the seventh embodiment. ; - ; ^ 
[01 27] Fig. - 12 is a time chart illustrating selections 
of the input process number, the audio process number, 
and the video process number in the buffer number set- 
ting section 903 of the embodiment. , , ; ^ 
[0128] When the reference, signal is input ^t time 
TO, the multiplexed data, of the 0-th frame begins to be 
input into sub-buffer 0 in accordance with the input proc- 
ess number 0. When the input is completed at time T1 . 
the input end signal is transmitted from the buffer to the 
buffer number setting section, "0", is set to the audio,, 
process number and the video process . candidate . 
number, and a number (in this case, **1") other than the 
video process number "0" and ^the audio process, 
nunrtoer **0" is set as a new i nput process number. At the 
same time, a coded audio is extracted from buffer O and, 
the audio decoding process is started. , . 
[0129] Furthermore, the video process candidate 
number "0" is irnmediately set to the video process 
number, the coded video is extracted from subrbuffer 0. 
and th^ video decoding process is started. - . - 
[0130] When the . next reference signal is input at 
time T2, the rnultiplexed data pf the first frame is input 
into sub-buffer 1 cpirespondihg to the input process 
number "l.'V • , , , 
[0131] The other pperations are identical with those 
of the seventh embodiment, 

[0132] . As V described above,, according to the 
embodiment also, a buffer for at least three frames is 
prepared, and the buffer management desci;ibed in the 
embodiment js conducted,. whe^reby video and audio, 
signals can be efficiently , decoded and. reproduced. 
When a reference signal is used as in the embodiment, 
particularly, the input of a multiplexed data is performed 
in the . unit prone frame. without delay, and abnormal 
reproduction due to. dete^^ audio decoding proc- 

- ess is prevented frcyn .cecums 

[0133] .. . , Jp Jhe sixth, seventh, and eighth embodi- 
ment, reproduction .ql only a video, or that of,prily an 
audio may, be perfprme^^^^ . 

[0134] "; t , R^^^ 1^ , is a view- illustrating a ninth embodi- 
ment of the invention. Jn.Fig. lS,. 1301 denotes a refer- 
ence signal, F^rpducipg section, .1302 denotes an audio 
data inputting section, i 303 denotes ari audio input con- 
frolling, section, ^.1 304 deriotes.^ number setting 
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section. 1305 denotes an audio buffer, 1306 denotes kn- > 
audio buffer selecting section. 1307 denotes an audio 
decoding section, 1308 denotes an: audio outputtingt:' 
section, and 1351 denotes a reference signal. . ; r u t 
[0135] In the embodiment, it is assumed that 
audio buffer has two sub-buffers 0 and 1 . and the audio 
inputting process and the audio decoding process are 
performed in different threads. 
[0136] Next, the operation of the embodiment wil! 
be described. io - 

[0137] The reference signal producing section 1301 
supplies the reference signal 1351 to the audio input 
controlling'section' 1302, once in one frame period (in 
the case of sin NTSC signal, about 1/30 sec, or more 
corr ctly 1001/30 msec,):! - ' is 

[0138] \Aftierithie reference signal 1351 is input, the 
audio input controlling section 1303 starts to obtain a 
coded audio data from the audio data Inputting section 
1 302, and the data is held by the sub^buffer in the audio 
buffer 1305 corresponding to the audio input number so 
1352 which is supplied from the buffer number setting 
section 1304. VVheh the input of a coded audio data of 
on frame is completed, the audio buffer 1305 transmits 
the audio input end signal 1353 to the buffer number 
setting section 1304. 2S 
[0139] In the buffer number setting section 1304, 
when the audio input end signal 1353 is received, the 
audio process number 1354 is immediately transmitted 
to the audio buffer selecting section 1306. The audio 
buffer selectirig seciidh 1306 extracts the coded audio so 
data from the sub-buffer iii the atjdio buffer 1305 and 
corresponding to the audio process number "1354,- and 
supplies the extracted data to the audio decoding sec- 
tion 1307. the audio decoding sectibn '1307 decodes 
the coded audio data, and supplies the d^oded data to 35 
the audio putputting section 1308 to perform audio 
reproduction. 

[0140] Fig. 14 is a time chart illustrating selections 
of the audio Input number and the audio process 
number in the buffer number setting section 1 304 of the 40 
embodiment. . ? 

[0141] When the reference signal Is generated at 
time TO. "0" is set as the audio input number, and the 
input of a coded audib data of a 0-^th frame is started. 
The destination of the input is sub-buffer 0 of the audio 45 
buffer ' - - ^ ■ ■ ' ■ 

[0142] When the audio input of the 0-th frame is 
ended at time T1 , "0" which is the audio input number is 
set to the audio process number, arid a nu 
than •'0," I.e., "1" is set as the audio input number. At the so 
same time/the coded audio data of the 0-th frairie is 
obtained from sub-buffer 0 correspbreiirig to the audio 
process number 0. and the audio decbdihg is startect 
[0143] When the reference signal is generated at 
time T2, a coded audio data of a first frame is input into ss"^ 
sub-buffer 1 of the audio buffer correspbriidihg to the 
audio input number "1 ' = ■ ' ' 
[0144] When a processor of Pentium or* higher 



which is mainly used as a CPU is enrployed, the audio 
decoding process forgone frame can be usually ended 
within a period which: is sufficiently shorter than one 
frame.period. Even if the decoding of a second frame is 
being performed when the input of the coded.audio of a 
third frame is ended at time T3. therefore, the decoding 
can be ended before the input of the next coded audio is 
started at time T4. so that the audio reproduction Is per- 
formed without producing a problem.* 
[0145] As^ descrit>ed above, , according to the 
embodiment, when ai buffer for at least two frames is 
prepared, the audio decoding is enabled in most cases 
without lacking any framei and reproduction can be per- 
formed without causing sound skipping. ,;: : , ' 
[0146] ^ Fig. 15 is a block diagram illustrating a tenth ; 
embodiment of the invention. In Fig. 15, 1501 denotes a / 
data inputting section. 1502 denotes an input controlling 
section; 1503 denotes an input suppress signal setting/ 
section. 1504 denotes a buffer, 1505 denotes a video 
process candidate list holding section; 1506 denotes a 
video decoding section. 1507 denotes a video output- 
ting section, 1551 denotes an input candidate frame 
number, 1552 denotes an input suppress signal, 1553 
denotes an input end signal, 1554 denotes a video proc- 
ess frame number, and 1555 denotes a decoding end ' 
signal, in the embodiment, the decoding unit is a frame, ; 
and the decoding unit number is the frame number. 
[0147] Next, the operation of the embodiment will 
be described. 

[0148] A coded video data which is input into the 
data inputting section 1501 is supplied to the input con- 
trolling section 1502 while the frame number is set as 
the input candidate frame number 1 551 . If the input sup- 
press signal 1552 supplied from the input suppress sig- 
nal setting section 1503 is OFF. the input controlling 
section inputs the coded video data of the input candi- 
date frame into the buffer 1503. 
[0149] When the input of the coded video data is 
completed, the input end signal 1553 Is supplied to the 
video process candidate list holding section 1504. The 
video process candidate list holding section 1504 con- 
trols the irput suppress 'Signal setting section 1503 in 
accordance with a video process candidate list held 
therein. 

[01 50] The video decoding section 1 506 receives a 
supply of a video process frame 1554 from the video' 
process candidate list holding section 1505. decodes 
the corresponding frame, and supplies the decoded 
data to the video outputting section 1507. When the 
video decoding is eiided, the decoding end signal 1555 
is supplied to the video process candidate list holding 
sectibn 1505. - - ^ ^ 

[0151] ' Fig. 16 is a time chart illustrating operations 
of the input candidate frame number, the video process 
candidate list, the input suppress signal, and the video 
process frame in the embodiment - . ■ 

[0152] • It is assumed that, before the start of the 
process, the video process candidate list is a null set {^) 
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and the input suppress signal: is OFF. ? . ^i! M - 
[0153] When a coded video data of a 0-th frame is* 
input into the inputting section at time TO, the input can- 
didate frame number "0" is obtained. Since the input 
suppress' signal iis OFF, the input controlling section 
starts the writing into the buffer at time T1. and the 
frame number of the next frame is set as the input can- 
didate frame number. ^ » - 
[0154] When the input is completed at time T2. the 
Input process frame number "O" the input of which has 
been completed is added to the video process candi- : 
date list. However, the top element of the video process - 
candidate list i e.; "0" is Jmmediately set as .the video, 
process frame and remoyed from the video proces^v 
candidate list.- and the v^video.. decoding process , is 
started: ..-.«■. >* . 

[01 55]' When the input of a first frame into the buffer 
is completed at time T3, "1" is added to the video proc- 
ess candidate list. Since the video decoding is being 
processed, however, the video process candidate list 
becomes {1}. " 
[0156] The video decoding of the O-th frame is 
ended at time T4. Therefore, is removed from the 
video process candidate liist and the list becomes <|>. 
[01 57] When the input of a third frame, is completed 
at time T5, the immediately previous video process can- 
didate list is {2}. and hence 3 is added to the rear so that 
the list is {2. 3}. At this time, the element number of the 
video process candidate list becomes 2, and the input 
suppress signal is therefore set to ON. 
[0158] At time :T6. the input candidate frame 
number is "4." At this time, the input suppress signal is 
ON. and hence the input of a fourth frame is not per- 
formed. 

[0159] When the decoding of the first frame Is 
ended at time T7. the decoding process of the second 
frame corresponding to **2" which is the top element of 
the video process candidate list is started, and "2" is 
removed from the video process candidate list. 
At the same time, since the video process candidate list 
from which "2" has been removed is {3} and the element 
number is 1 , the input suppress signal is therefore set.to 
OFF . 
[0160] As described above, according to, the 
embodiment, when the video decoding process time 
period is relatively longer than the input time period, the 
input into the buffer is suppressed according to the time 
difference between the inputting process and the 
decoding process, thereby ;allowing the CPUto allocate 
more resources to the video decoding.. As a result, the 
video reproduction rate can be improved. 
[0161] The decoding method of the invention can 
be realized as a decoding device, and also as a . pro- 
gram which operates in, a PC. This program can,,be ; 
recorded on a recording, medium such as ai.magnetic 
disk, and can be easily ported to, another PC. Further-, 
more, the program can be moved to^ another computer 
via a, network. „ <. , > 



Industrial Applicability / - 

[0162]^ J ■ As described above, according to the inven- 
tion, when a coded video or a coded audio is to b .. 

5 decoded by software by using a PC, the data input, the 

' audio decoding, and the video decoding can be 
decoded more efficiently, with the result that all audio 
data can be reproduced in real tirne and the video repro- 
duction rate can be improved. 

10 [0163] When the architecture (the performance of a 
CPU, the rate of a txjs. the form of the bus. and the like) 
of the executing PC is not lower in performance than 
that which is mainly^ used at present, the invention is 
effective, and its practice attains excellent effects., 

15 [0164] The coded audio signal in the invention 
includes a signal which is obtained by sampling an 
audio signal and simply rearranging the samples, and 
also a signal which is obtained by sampling an audio 
signal and further subjecting the samples, to high effi- 

20^: dency coding. . . . , ^ 

The decoding method of the invention cartvb.e realized 
also as a program which operates on a PC. This pro- 
gram can be recorded on a recording medium such as a 
magnetic disK and can be easily ported to another PC. . 

25 Furthermore, the program can be moved to another; 
computer via a network. ^ ^r- 

Claims ^ . , -.r 

30 1, A decoding method for reproducing by decoding a 
coded audio signal every each decoding unit.which 
is obtained by dividing said.cod^ audip signal in a 
direction of a time axis in audio decoding means, 
and by decoding a ; coded. yideo:^^^^^ 

35 decoding rneans. wherejri,, i^,, .^ , 

a decoding target time for each of the decoding 
units is determined. . : ,^ , > 
a reference time of a decoding process for 
40 - each of the decoding units is output, ; , 

in the decoding unit, the reference time is com- 
, pared with -the decoding , target time corre- 
sponding to the reference time, and. if the 
. ; ^ reference time is later. than the decoding target 
45 time, the process .of decoding the coded video 

signal in said video decoding means is tempo- 
. rarily interrupted. . . : , 

2. .A decoding method for reproducing by decoding a 
50 ..coded ajudio, signal every each decoding unit which 
is obtai^ed by dividing said coded audio signal in a 
directipn ;Of.^a,time axis in audio decoding rneans. 
and , by, decoding , a coded video signal in video ^ 
J , decoding r^eans.^-yv^ . . • . 

! ,1. a decodingiargettim each of the decoding 

units is determined. 
- . . a Teferj^nce time of a ciecoding, process for 
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each of thedecodirig units is output, - " 
in the decoding unit, the reference time Is com- 
pared with the decoding target time oorre- 
sponding to the reference time, and, if the 
reference time is earlier than the decoding tar- s 
get time, the process of decoding the coded 
audio signal in said audio decoding means is 
temporarily interrupted, and, if the reference 
time is later than the decoding target time, the 
process of decoding the coded video signal in io 
said video decoding means is temporarily inter- 
rupted. 

3. A decckiih^ nietho^ for reproducing by decoding a 
cdded ^udlo Sighai every each decoding unit which is 
is obtained by dividing said coded audio signal in a 
direction of a time axis in audio decoding means, 
and by decoding a coded video signal in video 
decodirig 'means, virherein 

• ' ' ' so 

a decoding tergjet time for each of the diacoding 
units ]s determined. 
• a reference tinii? "of a decoding process for 
each bf the deoodihg units is output, 
in* the decoding unit, the reference time is com- ss 
pared with the decoding target time corre- 
sponding to the refererice time, and, if the 
^ \ reference time is later than the decoding target 
' ' time, priority setting means sets a priority of 

said video decoding means to be lower, or a so 
- priority of said aijdio decoding means to be 
^higher ' - " " : 

4. A decoding method for reproducing by decoding a 
coded audio signal every each decoding unit which 35 
is obtained by dividing said coded audio signal in a 
direction of al tirrie axis in audio decoding means, 
and by decoding a coded video signal in video 
decoding means, wherein 

■ • ■ ~ " ■ ^ ^ . ■ c = • - ^ 

a decoding target time for each of the decoding 
units is determined, ■ ' 

a reference time of a decoding process for 
each of the decoding units is outjaut, * 
in the decoding unit, the reference tirhe is com- 45 
pared with* the decoding target time corre- 
sponding to the reference time, and. if the 
refeirence time is earlier than the decoding tar- 
get time, priority setting means sets a priority of 
said Video decoding means to be higher, or a so 
(iriority of said audio debcxiing- rneans to be 
tower, and, if the reference tirriie is later than the 
decoding Wget time, said pribrit^^^ 
means iets the priority'of said video decoding 
means to be made lower, or the prioriti^'df said ss 
audio decodirig means to be' higher.' - ■ = 

5. A decoding method according tb ahy-bne'b^^^^^ 



1 to '4/ characterized in that the reference tiriie is a 
start tiriie br an end time of a decoding of the 
decoding unit. . 

6. A decoding method iaccordihg to any one of claims 
1 to 5; characterized in that the decoding unit is one 
frame of a video signal.- ' ^ 

7. A decoding method for reproducing by decoding a 
coded video data in a decoding unit, characterized 
in that- " -^^"^ ' ^' ■ ' - '-v \ ■ - 

\r\ the c^e wh^r^ the coded tvideb data in the 
' ' decodirig'uhit is ihputfthe coded An^ is 
held in the decodihg uriit by 'Srr n of 
buffers Xh 1s a'fiatural hyrnbef^); a aided video 
' data is selected from said buffers, and a decod- 
ing is performed by a video processing section, 
when an input, a selection ^ arid a decoding are 
performed on the buffers while using: an input 
process number indicating into which one of 
said buffers a coded video data is to be input; a 
video process number iridicating from which 
one of said buffers a coded video data that is 
being decoded is selected; and a video proc- 
ess candidate number indicating a buffer which 
stores a coded video data that is to bid selected 
after the video process is erxled, 
the coded video data in the decoding unit is 
input by Input controlling means into a buffer 
corresponding to the input process number, 
when the video process is ended, the video 
process candidate number at the ending timing 
is set to a next video process number, and a 
coded video data of a buffer corresponding to 
the video process candidate number is 
decoded, and 

when an input in the decoding unit into said 
buffer is ended, an input process number corre- 
sponding to a buffer in which the input has 
been just ended is set to the video process 
candidate number, and a number other than 
the video process candidate number to which 
the input process number has been just set, 
and other than the video process number at the 
erKfing timing is set as the new input process 
number. : 

8. A decodii-ig method for reproducing by decoding a 
coded audio data and a coded video- data in a 
decoding unit, characterized in that 

in the case where a multiplexed data in which 
' "the coded video data in the decoding unit and 
' ■ the coded audio data in the decoding unit are 

rnuttiplexed is input, the multiplexed data is 
'* held iri the decdding unit by an n number of 
' - buffers (n is a naiural number), a coded video 
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data and a coded audio data are selected from 
said buffers, and a decoding Is performed by a 
video processing section and an audio 
processing section, 
. when an input, a. selection,- and a decoding are 
performed on the buffers while using: an input 
process number indicating into which one of 
said buffers a coded video data and a coded 
audio data are to be input; a video process 
number indicating from v/hich one of said buff- 10 
ers a coded video data that is being decoded is 
selected; and a video process candidate 

. numbeir ' indicating a buffer which stores a 
; codjesd video data that is.to be sele^ 

video process is ended, ^ , J : - % is 

: the multiplexed data in the decoding unit Is 
input by Jnput controlling means, into a buffer 
corresponding to the Input process number, 

' after the input of the rnultiplexed data into, said 
buffer Js ended, the coded audio data is 20 

, decoded by said audio processing section, 
when a video process is ended, the video proc- 
ess candidate number at the ending timing is 
set to a next video procass number, and a 
coded video data of a buffer corresponding to 2S 
the video process . candidate number is 

- decoded, and 

when the audio decoding process is ended* an 
. input process number corresponding to a buffer 
in which the audio decoding process has been 30 
just ended is set to the video process candidate 
number, and a nuniber other than the video 
process candidate number to which the input 
process number has been just set, and other 
than the video process numt>er at the ending 3S 
timing is set as the new input process number. 

9. A decoding method for reproducing by decoding a 
coded audio data and a coded video data in a 
decoding unit, characterized in that 4c? 

in the case where a multiplexed data in which 
the coded video data in the decoding unit and 
the coded audio data in the decoding unit are 
multiplexed is input, the multiplexed data is 45 
held in the decoding unit by an n number of 
buffers (n is a natural number), a coded video 
data and a coded audio data are selected from 
said buffers, and a decoding is performed by a 
, video processing section and an audio so 
processing section, - ; 

when an input, a selection, and a decoding are 
performed on the buffers while using: an, input 
rprocess number indicating into which one of 
said buffers a coded rvideo data and a coded 55 
' audio, data are to be input; a video process 
number indicating from which one of said buff- 
- : ; ers a coded video data that is f>eing decoded is 
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selected; an audio process nurriber iridicating 
.,,. .fron:> which one of said buffers a coded audio 
/.'i J -data th^ is being decoded is selected; and a 
.. .^ video process candidate number indicating a 
r buffer which stores a coded video data that is to 
, jbe selected after the yideo process is ended, 
the, rnultiplexed data in the decoding unit is 
input by input controlling means into a buffer 
corresponding to the input process number. 
. after the input of the multiplexed data into said 
buffer is ended,, the coded audio data is 
decoded by said audio processing section, 
when a video process is ended, the video proc- 
ess candidate number at the ending tirrTiing is 
■ ^ ^ set to a next video process number, and;, a 
coded video data of a buffer corresponding to 
the video process candidate nunrtoer ^ is 
decoded, and 

when an input in the decoding .unit to, said 
buffer is ended, an input process number corre- 
sponding to a buffer in which the, input has 
been just ended is set to the au^ip process 
number and the video process ^ candidate 
number, arxl (1) a number other ^hari .the audio 
process number to which the. input, process 
nunrrt)er has been just set. andotheMhan the 
videooprocess number at, the input ending tim- 
ing, or (2) a nunrtber ottier than the video proc- 
ess candidate number- to ■ v^ich tfie, input 
process number, has been just , set, and other 
than the video process, nurhber at the input 
process timing is set as the new input process 
number 

10. A decoding njethod ac9prding to, claim 9, character- 
ized jn that the muftiptexed data to be input into said 
buffer is input in accprdance with a predetermined 
. reference signal.! . . , - ^ 

11. A decoding method for reproducing by decoding a 
coded audio data in.^ari audio - decoding unit, 
wherein 

in the case where the coded audio data in the 
. audio, deccding.unitis heild by^ an ^^^^ number of 
-buffers (ni, is a natural number), the. coded 
^ audio dat^ is sejlected from said buffers, and a 
/decoding is performed by an audio processing 
section, y ;■• ; w,, ■ • • - ■ .,■ * . 
when an input, a selection,-and a decoding are 
, f perforrned pn.the buffers while usirig: an input 
^process nunTber, indica^^^ one of 

, said buffers . a xoded.a^ data is to be input; 
and* an^ audio f^^^^ ir>dicating from 

: V » ' V'i.hic^^^^ said buffers a coded audio data 
that is being decoded is selected, 
when an audio reference signal which is pro- 
^^.j ..dMCjSi^^ to the 
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audio decoding unit is produced^ ah ' audio 
buffer con^esponding to the audio input number 
is caused to start an input of the coded audio 
data, . , 

the coded audio data is selected from an audio 
buffer of the audio process number, the coded 
audio data is decoded by said audio process- 

' ing section. * 
when a coded audio input in the decoding unit 
is ended, an audio input number corresponding 
to a buffer in which the input' has been just 
ehdeid is set to the audio process number by 
process nlimber updating means, and in place 
a number other than; the audio process number 

' to which the audio Input number has been just 
set is set as the hew audio input number. 

12/ A decioding method for input and reproducing by 
decoding a coded data in a decoding unit, wherein 

in the case where the coded data in the decod- 
. ing unit is held by ah m number of buffers (m is 
a natural numt^i-). the coded data is sielected 
from said buffers, and a decoding is performed 
. by a processing section. 
^ when an input of a coded data into said buffers 
is ended; numbers of buffers in which the input 
\ :is ended are accuniulated into a process candi- 
' date list.; when a coded data is selected for 
decoding from a predetermined buffer, a 
nurnber of the selected buffer is deleted from 
the process bartdidate list; ; 
when a process of a coded data is ended. 
; among buffei' nurnbers accumulated in the 
process caindidate list, a buffer number an 
accumulation tirhe of which is earliest is 
selected, and a decoding Is performed by a 
processing section, and ' 
when th^ cod^ data is to be input a number of 
buffer numbers which are abcumulated in the 
process candidate list Is checked; and, if the 
number reaches a predetermined number a 
which is not larger than the number m of said 
buffers, the coded data i^ hot irif^ut, and, if only 
buffer nijnibers a number of which is smaller 
than a are accumulated, the coded data is 
jnput. , " " ■ '. ^ . 

13. A decoding method according to of clairh 7, 8, 9. or 
10. characterized in that the decodifig ijnit is one 
frame of a video. / / ^ ' 

14., A decoding method acbbrdirig to of clairti 7 9. or 
10. characterized in that n is a natural n^ 
which is equal to or larger than 3. \ 

15. A decoding method accpi'ding td ordaim 11. char- 
acterized in that m is a natural hOrtbef which is 



equal to of larger than 2.- ^ 

16. A decoding method according to of claini 12, char- 
acterized in that the coded data is a coded video 

5 data, and the decoding unit is one frame of a video. 

17. A decoding method according to of claim 12, char- 
acterized in that a' is a natural number which is 
equal to or larger than 2. ' ' 

10 ■ " " ' ■ ' ' " • ' ' ' 

18. A' program recording mediufh characterized in that 
a program for realizing all or a part of steps of the 
decoding method according to any -one* of clairns 1 
to 17 by a computer is stored in said medium. 

Amended claims under Art.19.1 PCt • 

1. (Amended) A decoding rhethod for reproducing 
by decoding a coded audio signal every each 
20 decoding unit which is obtained by dividing said 
coded audio signal in a direction of a'time axis in 
audio decoding means, and by decoding a coded 
video signal in video decoding means; wherein 

25 a decoding target time for each of the decoding 

units is determined, 

a reference time which is a time when a decod- 
ing process for each of the decoding units is 
started or which is a time when the decoding 
30 process fbr each of the decoding units is ended 

is output, 

in the decoding unit, the reference time is com- 
pared with the decoding target time corre- 
sponding to the reference time, arKJ, if the 
35 reference time is later than the decoding target 

time, the process of decoding the coded video 
signal in said video decoding means is tempo- 
rarily interrupted. 

40 2. (Amended) A decoding method for reproducing 
by decoding a coded audio signal every each 
decoding unit which is obtained by dividing said 
coded audio signal in a direction of a time axis in 
audio decoding means, and by decoding a coded 

45 ' video signal in video decoding means, wherein 

a decoding target time for edch of the decoding 
units is determined. 
' a reference time which is a time when a decod- 
50 ing process for each of the decoding units is 

started or which is a time when the decoding 
process for each of the decoding units is ended 
^ " is output, ' ' ' 

' in the decbding unit/the reference time is com- 
55 pared with the decoding target time conre- 

sponding to the reference time, and, if the 
reiference time is earlier than the decoding tar- 
get time, the process of decoding the coded 
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audio signal in said audio decoding means is 
temporarily interrupted, and, if the reference 
time is later than the decoding target time, the . 
process of decoding the coded video signal In 
said \rtdeo decoding means is temporarily inter- 5 
rupted. 

3. A decoding method for reproducing by decoding 
a coded audio signal every each decoding unit 
which is obtained by dividing said coded audio sig- 10 ^ 
nal in a direction of a time axis in. audio decoding 
means, and by decoding a coded video signal in 
video decoding means, wtierein r^^^ , 

a decoding target time for each of the decoding is. 
units is deterrnined, . j . 1^ . % . 

a reference time of a decoding process for 
each pf the decoding units is output, 
in the decoding unit, the reference time is com- 
pared with the decoding target time.corre- 20 
spending to the reference time, and,, if the 
reference time is later than the decoding target 
tirne, priority setting means sets a priority of 
said video decoding means to be lower, or a 
- . priority of said audio decoding means to be 25 
higher, 

4. A decoding method for reproducing by decoding 
a coded audio signal every each decoding unit 
.which is obtained by dividing said coded audio sig- 30 
nat in a direction of a time axis in audio decoding 
means, and by decoding a coded video signal in 
video decoding means, wherein 

a decoding target time for each of the decoding 35 
units is determined, 

a reference time of a decoding process for 
each of the decoding units is output, 
in the decoding unit, the reference time is com- 
pared with the decoding target time corre- 40 
spending to the reference time,' and. if the 
reference time is earlier than the decoding tar- 
get time, priority setting means sets a priority of 
said video decoding means to be higher, or a 
.priority of said audio decoding means to be 45 
lower, and, if the reference time is later than the 
. decoding target time, said priority setting 
means sets the priority of said video decoding 
means to be made lower, or the priority of said 
audio decoding means to be higher. 50_ 

5. (Amended) A decoding method according to 
ciaim 3 or 4. characterized in that the reference 
time is a start time or an end time pf a.decoding of 

the decoding unit. . i , . ss ^ 

6. A. decoding method , according to any _one of 
claims 1 to 5,.charaderized in that, the decoding 



. ^unit iS:One , frame of a video signal. 

e ^7. (AmerKled) A method for reproducing by decod- 
ing a coded video data in a decoding unit, charac- 
terized in that ^ ^ , 

in the case where the coded video data in the 
decoding unit is input, the coded video data is 
held in the decoding unit by an n number of 
buffers (n is a natural number), a coded video 
data is selected from said buffers, arid a decod- 
. . ing is performed by a video processing section. 
, when an input, a selection.' and a decoding are 
performed on the buffers while usiiigi an input 
process number indicating into which one of 
said buffers a coded video data is to be input, a 
video process number indicating from which 
one of said buffers a coded video data that Js 
being decoded is selected;. arid a, video proc- 
ess candidate number indicating a buffer which 
stores a coded video data that Is to be selected 
after the video process is ended. , 
the coded yidep data in the decbd.ing unit is 
input by input, controlling means into a buffer 
corresponding to the input process number, 
when the video process is ended, the video 
process candidate number at the ending timing 
is set to a next video process number, and a 
coded video data of a buffer corresponding to 
the video process candidate .nurtiber is 
decoded, and . \ 
when an input in the decoding unit into said 
buffer is ended, an inputprdcess number conre- 
spondihg to a buffer iri which the input has 
been jgst ended 'is set to the video process 
candidatie, number, and a number other than 
' the video process candidate nunriber to which 
the input process number has been jiist set. 
and other than the video process number at the 
ending timing is set as the^ new input process 
number ^ ^ , / 

8. (Amended) A diecoding method for reproducing 
by decoding a coded audio data and a coded video 
.data in a decoding unit, characterized in that 

in the case where a multiplexed data in which 
the coded video data in the decoding unit and 
the coded audio data in the decoding unit ar 

. multiplexed is input, the multiplexed data is 
held ih the decoding unit by an n number of 
buffers (n is a natural number), a coded video 

. data, and a coded audio data are selected from, 
. , / said buffers, an^ is performed by a 

video ^proci^sihg section and an audio 
processing section, 
, , vyhen an jnput, a selectiori. and a decpding are 
perforriied on the buffers while using: an input 
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process number indicating into which drie of 
said buffers a coded video data and a coded 
audio data are to be input; a video process 
number indicating from which one of said buff- 
ers a coded video data that is being decoded is 5 
selected; and a video process candidate 
number indicating a buffer which stores a 
coded video data that is to be selected after the 
video process is ended, 

the multiplexed data in the decoding unit is 10 
input by input controlling means into a buffeV 
conresponding to the input process number, 
after the input of the multiplexed data into said 
buffer is ended, the coded audio data is 
decoded by said audio processing section, is 
when a video process is ended, the video proc- 
ess candidate number at the ending timing is 
set to a next video process number, and a 
coded video data of a buffer corresponding to 
the video process candidate number is 20 
decoded. arxJ 

when the audio decoding process is ended, an 
input process number corresponding to a buffer 
in which the audio decoding process has been 
just ended is set to the video process candidate 25 
number, and a number other than the video 
process candidate number to which the input 
process number has been just set. and other 
than the video process nunnber at the ending 
timing is set as the new input process number. 30 

9. (Amended) A decoding method for reproducing 
by decoding a coded audio data and a coded video 
data in a decoding unit characterized in that 

35 

in the case where a multiplexed data in which 
the coded video data in the decoding unit and 
the coded audio data in the decoding unit are 
multiplexed is input, the multiplexed data is 
held in the decoding unit by an n number of 40 
buffers (n is a natural number), a coded video 
data and a coded audio data are selected from 
said buffers, and a decoding is performed by a 
video processing section and an audio 
processing section, 45 
when an input, a selection, and a decoding are 
performed on the buffers while using: an input 
process number indicating into which one of 
said txiffers a coded video data and a coded 
audio data are to be input; a video process so 
number indicating from which one of said buff- 
ers a coded video data that is being decoded is 
selected; an audio process number indicating 
from which one of said buffers a coded audio 
data that is being decoded is selected; and a ss 
video process candidate number indicating a 
buffer which stores a coded video data that is to 
be selected after the video process is ended. 



the-muttipl©(ed data in the decoding unit is 
input by input controlling means into a buffer 
correspbndihg to the input process number, 
' after the input of the multiplexed data into said 
buffer is ended, the coded audio data is 
decoded by said audio processing section, 
when a video jDrocess is erided. the video proc- 
^ ess candidate riurhber at the ending tirhing is 
set to a next video process hurnber. and a 
coded vided dafa of a buffer corresponding to 
the video process candidate number is 
' ' decoded, and . . ^ 

^ whSn ain input In the decodihgoinit to said 
buffer is ended, an input process number corre- 
sponding to a buffer in which the input has 
' been just ended is set to the audio process 
number and the video process ' candidate 
number, and (1) a number other than the audio 
process number to which the input process 
number has been just set, and other than the 
video process number at the input ending tim- 
ing, or (2) a number other than the video proc- 
ess candidate number to which the input 
process number has been just set, arid other 
than the video process number at the input 
process timing is set as the new input process 
number. 

10. (Amended) A decoding method according to 
claim 9. characterized in that the multiplexed data 
to be input into said buffer is input in accordance 
with a predetermined reference signal. 

11. (Amended) A decoding method for reproducing 
by decoding a coded audio data in an audio decod- 
ing unit, wherein 

in tfie case where the coded audio data in the 
audio decoding unit is held by an m number of 
buffers (m is a natural number), the coded 
audio data is selected from said buffers, and a 
decoding is performed by an audio processing 
section. 

when an input, a selection, and a decoding are 
performed on the buffers while using: an input 
process number indicating into which one bf 
said buffers a coded audio data is to be input; 
and an audio process number indicating from 
which one of said buffers a coded aiidio data 
that Is being decoded is selected, 
when an audio reference signal which is pro- 
duced at time intervals corresponding to the 
audio decoding unit is produced, ah audio 
buffer corresponding to the audio input number 
is caused to start an input of the coded audio 
data^^ _ 

the coded audio data is selected from an audio 
buffer of the audio process number, the coded 
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audio data is decoded by said audio process- 
ing section, - * . : . , 
when a coded audio input in the decoding unit 
- is ended, an audio input number con-esponding 
to a buffer in which the input has been just s 
ended is set to the audio process number by 
process number updating means, and in place 
a number other than the audio process number 
to which the audio input r}umt)er has been just 
set is set as the new audio input number, io 

12. (Amended) A decoding method for input and 
reproducing by decoding a coded data in a decod- 
ing unit, whprein , • . , , 

in the case where the coded, data in the decod- 
7 ing unit is held by an m number of buffers (m is 
a natural number); the coded data is selected 
. from said buffers, and a decoding is performed 
by a processing section. 20 
when an input of a coded data. into said buffers 
is ended, numbers of buffers in which the input 
is ended are accumulated irito a process candi- 
date list, when a coded data is selected for 
decoding from a predetermined buffer, a 25 
, number of tiie.selected buffer is deleted from 
the process candidate list, 
when a process of a coded data is ended, 
among buffer numbers accumulated in the 
process candidate list, a buffer number an 30 
accumulation time of which is earliest is 
selected, and a decoding is performed by a 
processing section, and 
when the coded data is to be input, a number of 
buffer numbers which are accumulated in the 35 
process candidate list is checked, and, if the 
number reaches a predetermined number a 
which is not larger than the number m of said 
buffers, the coded data is not input, and, if only 
. buffer numbers a number of whjch is smaller 40 
tiian a are accumulated, the coded data is 
input. , 

13. (Amended) A decoding method according to of 
claim 7. 8, 9, or 1 0, characterized in that the decod- 45 

, ing unit is one frame of a video. 

14. (Arriended) A decoding method according to of 
-claim 7.. 8, 9, or 10. characterized in that n is a nat- 
ural number which is equal to or larger -than 3. so 

. 15 .(Amended) A decoding method according to of 
claim 11, characterized in that m. is'^ a natural 
number which is equal to or larger than 2. , 

' 16. (Amended) A decoding metiiod according to of 
claim 12,. characterized in that.the coded data is a 
. coded, video data, and the decociing unit is one 



frarrie of a video. . 

^7. (Amended) A decoding method according to of 
claim 12, characterized iri that a is a natural.number 
which is equal to or larger than 2, 

,18. (Amended) A program recording medium char- 
acterized in that a program for realizing all or a part 
of steps of the decoding method according to any 
one of claims 1 to 1 7 by a computer is stored in said 
medium. 
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